Bi-directional tap communication device

ABSTRACT

Data receiving systems, methods and computer products are provided that include actuators in mechanical communication with fingers. Each of the actuators is arranged to apply a mechanical stimulus to a respective finger.

BACKGROUND Field

Example aspects described herein related generally to chorded data entrysystems, and more particularly to a bi-directional arbitrary surface andfinger position keyboard.

Description of Related Art

Even before the invention of the typewriter in 1868, efforts have beenmade to use finger combinations in a “chorded” arrangement to enteralphanumeric information. FIG. 1 shows an example of a Baudot Keyboard100, the earliest five key chorded arrangement, which was invented in1870 by Jean-Maurice-Emile Baudot as a five key alphanumeric entrysystem for telegraph operators. Using only five keys, the operator couldenter 32 unique ‘codes’, which were mapped to corresponding alphanumericsymbols. Such codes were sometimes referred to as Baudot Codes, theprecursor of ASCII codes and the source of the term “baud.”

Other common examples of chorded keyboards are stenographic keyboards,which typically include 22 keys, and Braille keyboards, which typicallyinclude six or eight keys.

With the advent of computers, a number of attempts have been made to usechorded keyboards (sometimes referred to as “keyers”) for single-handeddata entry.

Touch screens have allowed “soft” keyboards to be incorporated intomobile devices. In addition to standard alphanumeric keyboards, a numberof attempts have been made to create chorded keyboards using touchscreen systems. While the above devices provide one-handed methods forcharacter entry, they all require that the operator contact a specificpoint, or set of points, with a specific finger, or set of fingers. This“aiming” requirement significantly reduces the speed and ease with whichdata can be entered.

A different strategy is employed by gesture-based systems, which enablean operator to enter data (and “point”) on a computer or mobile devicewithout employing a standard keyboard. In this context, a gesture is amotion, sensed by a computer, whereby a user moves a hand or finger froma starting point to an end point.

While gesture-based systems enable a user to enter characters withouthaving to aim for a specific point, typically they require that themotion have a trajectory from a certain starting point to a certainending point. Thus, to create a gesture, the user must necessarilytravel a minimum distance from one point to another. This is unlike atypical physical keyboard, where the only data that must be derived isbased on which key is pressed. Because of this, gesture-based systemsare significantly slower than physical keyboards.

SUMMARY

In accordance with example aspects herein, the foregoing shortcomingsare overcome by a novel, one-handed, chorded data entry system thatenables an operator to tap a combination of fingers on a surface,thereby causing a character to be entered in an electronic device. Thesystem allows the operator to tap on any surface, with the fingersarranged in any position. Neither the fingers nor the fingertips of theoperator need be aimed at any target. As a result, the system describedherein reduces finger travel and facilitates much faster and moreaccurate data entry than the prior systems.

In one embodiment, a data entry system includes a sensor apparatusarranged to generate a signal representative of a contact of one or morefingers singly or in combination against an arbitrary surface. Thearbitrary surface does not generate the signal.

The contact of one or more fingers may represent a finger combinationthat maps to any one of a character and a command, or a combination ofboth the character and the command.

The data entry system can also include a wearable structure constructedto support the sensor apparatus on any one of i) a dorsal surface of ahand ii) a palmar surface of a hand, iii) a wrist, and iv) phalanges ofa hand, or any combination of i), ii), iii) and iv).

In another embodiment, the data entry system further includes a memory.and the sensor apparatus further includes an image sensor arranged toacquire the signal, the signal being an image of the one or morefingers. The memory is operable to store the image.

The data entry system may also include a processor, communicativelycoupled to the sensor apparatus, operable to filter the image toascertain a location of each finger at the moment of contact against thearbitrary surface.

In yet another embodiment, the sensor apparatus further includes atleast one mechanical sensor arranged to acquire the signal, the signalcorresponding to a mechanical quantity caused by the one or more fingerscontacting the arbitrary surface.

The data entry system may also include a processor operable to collectthe signal received from the sensor apparatus at the moment of thecontact against the surface.

A data entry method is also provided including the steps of generating asignal representative of a contact of one or more fingers singly or incombination against an arbitrary surface. The arbitrary surface does notgenerate the signal.

The data entry method can also include mapping a finger combination thatis represented by the contact of the one or more fingers represents toany one of a character and a command, or a combination of both thecharacter and the command.

In one embodiment, the data entry method can also include acquiring theimage signal by an image sensor, the signal being an image of the one ormore fingers, and storing the image in a memory.

The data entry method may also perform filtering the image to ascertaina location of each finger at the moment of contact against the arbitrarysurface.

In yet another embodiment, the data entry method can include acquiringthe signal, by at least one mechanical sensor, the signal correspondingto a mechanical quantity caused by the one or more fingers contactingthe arbitrary surface.

The data entry method may further include collecting the signal receivedfrom the sensor apparatus at the moment of the contact against thesurface.

In yet another aspect, a non-transitory computer readable storage mediumstoring a computer program which when executed by a computer causes thecomputer to execute a method of data entry according to the methodsdescribed herein

In another embodiment, a data receiving system is provided. The systemincludes a plurality of actuators in mechanical communication with aplurality of fingers, each of the plurality of actuators arranged toapply a mechanical stimulus to a respective one of the plurality offingers.

In one embodiment, the mechanical stimulus of the plurality of actuatorsmaps to at least one of an alphanumeric character, a word, a phrase, anda sentence.

The data receiving system can also include a wearable structureconstructed to support the plurality of actuators on a plurality ofphalanges of a hand.

In yet another embodiment, the data receiving system includes a memory,a receiver operable to receive data, a decoder, communicatively coupledto the receiver, operable to decode the data into one or more tap codes;and a processor, communicatively coupled to the decoder, operable toactuate each of the plurality of actuators according to the tap codes.

A data receiving method is also provided. The method performs the stepsof applying a mechanical stimulus to a plurality of fingers using aplurality of actuators in mechanical communication with respective onesof the plurality of fingers.

In another embodiment, the data receiving method includes the step ofmapping the mechanical stimulus of the plurality of actuators maps to atleast one of an alphanumeric character, a word, a phrase, and asentence.

In yet another embodiment, the data receiving method includes the stepof supporting the plurality of actuators on a plurality of phalanges ofa hand using a wearable structure.

Another embodiment of the data receiving method includes the steps ofreceiving data; decoding the data into one or more tap codes; andactuating each of the plurality of actuators according to the tap codes.

Further features and advantages, as well as the structure and operation,of various example embodiments of the present invention are described indetail below with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The features and advantages of the example embodiments presented hereinwill become more apparent from the detailed description set forth belowwhen taken in conjunction with the drawings.

FIG. 1 illustrates an example of a Baudot Keyboard.

FIG. 2 illustrates an example system that may be employed in accordancewith various example aspects herein.

FIG. 3 illustrates an example of a camera-based sensor apparatus.

FIG. 4 illustrates another example of a camera-based sensor apparatus.

FIG. 5 illustrates another system that may be employed in accordancewith example embodiments herein.

FIG. 6 illustrates an example mechanical sensor apparatus.

FIG. 7 is a flowchart illustrating an example procedure that may beemployed according to various example embodiments herein.

FIG. 8 illustrates an example of an acquired image and a correspondingedge-detected image in accordance with various example aspects herein.

FIGS. 9 and 10 illustrate an example apparatus having both sensors andactuators in accordance with an example embodiment.

FIG. 11 illustrates example use cases in accordance with the exampleembodiments described herein.

DETAILED DESCRIPTION

The example embodiments of the invention presented herein are directedto methods, systems and computer program products for a chorded dataentry system that enables an operator to tap a combination of fingers onan arbitrary surface thereby causing a character or command to beentered in an electronic device, which are now described herein in termsof example one-handed chorded data entry systems. This description isnot intended to limit the application of the example embodimentspresented herein to one-handed use cases. In fact, after reading thefollowing description it will be apparent to one skilled in the relevantart(s) how to implement all of the following example embodiments using apair of the chorded data entry systems for use with an operator's lefthand, right hand, or a combination of both. In addition, the term“finger combination” as used herein refers to any one finger or acombination of fingers.

The example chorded data systems described herein allow an operator'sfingers to be arranged in any position and neither the fingers nor theirfingertips need be aimed at any particular target. As a result, thesystem reduces finger travel and facilitates fast and accurate dataentry. In addition, the system enables a user to enter characters orcommands into a computing device by tapping a combination of fingers onany surface.

FIG. 2 illustrates an example system 200 that may be employed inaccordance with various example aspects. System 200 includes a sensorapparatus 201 that is coupled to a processor 202 which in turn iscoupled to a transmitter 203.

In some example implementations sensor apparatus 201 operates as does acamera to the extent it can detect and convey information thatconstitutes an image of an object. In this implementation, the objectsare fingers of a hand and a surface that the fingertips or substantiallythe fingertips of the fingers contact. Sensor apparatus is arranged tohave the fingers in its field of view. Accordingly, the sensor apparatus201 may be considered a special purpose camera or a camera that has beenpurposed and arranged for the particular applications described herein.For simplicity, in this embodiment the terms “sensor apparatus 201” and“special purpose camera” may be used interchangeably herein.

Particularly, sensor apparatus 201 may be integrated into a wearablestructure such that the sensor apparatus 201 is located at the bottom ofa wrist or palm of a user (also commonly referred to as the dorsalsurface of a hand or palmar surface of a hand, respectively). In thisembodiment, sensor apparatus 201 is arranged in a manner that allows allof the fingertips of the user to be within its field of view. The othercomponents of system 200 including the processor 202 and transmitter 203can be located elsewhere or inside on the housing or structuresupporting the sensor apparatus 201.

In one embodiment, as sensor apparatus 201 captures finger combinationsbeing used with each tap, processor 202 performs instructions whichcause the sensed finger combination to be mapped to an appropriatecharacter or command. The transmitter 203 then transmits the mapped ordecoded character to a computing device via a wireless or wiredinterface.

Instead of determining finger combinations within system 200 byprocessor 202, in an alternative embodiment, processor 202 causes theimage data obtained from the sensor apparatus 201 to be transmitted tothe computing device for further processing (i.e., to process and filterthe images, determine sensed finger combinations and map the sensedfinger combinations to the appropriate character) via a wireless orwired interface. This would allow, for example, for the processingdevice of the computing device (e.g., in a smartphone or other computingdevice) to determine which fingertips were in contact with a surfaceduring the tap, and which were not. Such an arrangement can reduce thecomplexity and power requirement of processor 202.

In yet another alternative embodiment, processor 202 performs someprocessing on the captured finger combinations, such as by performingedge detection filtering and landmark identification from the image dataacquired by the sensor apparatus 201, and then forwarding the partiallyprocessed data through the transmitter to the computing device via awireless or wired interface for further processing by the computingdevice and identification of the tapped finger combination.

The sensor apparatus 201 need only detect which fingers are in contactwith the surface at the time of the tap, thereby allowing the user totap on any surface, with fingers arranged in any position. The user neednot aim at a specific key or location, or start the tap from a specificpoint (unlike in a gesture-based system), thus making the data entryprocess fast and reliable. Also, system 200 is mobile because tappingcan be done on any surface, such as a table, a leg of the user, a chair,and/or the like.

Sensor apparatus 201 may optionally include a module, such as amechanical sensor (e.g. accelerometers, vibration sensors, tactilesensors, force sensors, pressure sensors, gyroscopes, and the like; notshown), for sensing when a tap occurs (i.e. when any of the fingers ofthe hand have generated an acceleration, vibration, pressure or forceindicative that they have made contact with a surface). In an exampleembodiment, processor 202 senses that a tap has occurred by reading themechanical sensor, at which point the sensor apparatus 201 captures animage (or a series of images), and the processor 202 determines whichfingertips were in contact with a surface during the tap, and which werenot. The optional mechanical sensor can be located within the samehousing as the sensor apparatus 201.

In an example embodiment of this optional arrangement, system 200monitors the outputs of the mechanical sensor to determine that whethera tap (e.g., a contact of a finger with a surface) has occurred. Thebeginning of a tap event, in this situation, is indicated by a change inoutput from one or more mechanical sensor above a predeterminedthreshold. Once a tap is sensed, processor 202 causes an image sensor(discussed in more detail below) to capture image(s). In thisembodiment, the mechanical sensor can be located anywhere on the hand orwrist that allows the mechanical sensor to sense mechanical quantitiesof any of the fingers used for tapping. For example, the mechanicalsensor can be arranged to be located on the back of the wrist or hand,the palm, or on, for example, the proximal phalanges. Upon sensing acontact with a surface (i.e., a tap), the mechanical sensor generatesmechanical quantities in the form of a signal capable of being processed(e.g., compared against a threshold) by processor 202.

FIG. 3 illustrates an example camera-based sensor apparatus 300, orspecial purpose camera, which, in some example embodiments may furtherrepresent the sensor apparatus 201 that is described above in connectionwith FIG. 2. In one embodiment, sensor apparatus 300 includes an imagesensor 301 which is sensitive to light in the near infrared spectrum,with associated drive electronics and image processing means (notshown), and a wide angle lens 302 arranged such that all five fingersare in its field of view. The image sensor 301 generates a signalrepresentative of the image it captures.

Sensor apparatus 300 can also include an optical illumination systemincluding one or more infrared light emitting diodes (IR LEDs) 303 whichare strobed. In one embodiment, IR LEDs 303 are strobed insynchronization with the image sensor 301 or are illuminated wheneverthe device is turned on. In another embodiment IR LEDs 303 are strobedwhen a tap is detected.

An optical filter 304 is situated in front of image sensor 301 toprevent light having wavelengths that are below the infrared spectrum(e.g. 800 nm) from impinging on image sensor 301. Sensor apparatus 300may also include a polarizing filter 305 placed in front of the IR LEDs303 and lens 302, thus reducing the amount of ambient light that reachesthe image sensor 301.

In this example implementation, as shown in FIG. 3, sensor apparatus 300is implemented by attaching at least the sensor apparatus 300 to thebottom of a wrist of a user, arranged in a manner such that all of thefingertips of the user are within its field of view. As noted above inconnection with FIG. 2, sensor apparatus 300 also can be implemented atother locations, such as by locating the sensor apparatus 300 at thebottom of a palm of a user.

Referring to FIG. 4, in another embodiment, the optical illuminationsystem can operate in the visible light spectrum (e.g., 400-700 nm).This embodiment is similar to the above described IR embodiments;however the illumination system emits light in the visible spectruminstead of the infrared spectrum. FIG. 4 illustrates an examplecamera-based sensor apparatus 400, or special purposed camera, which, insome example embodiments may further represent the sensor apparatus 201that is described above in connection with FIG. 2. In this embodiment,the sensor apparatus 400 includes an image sensor 401 which is sensitiveto light in the visible spectrum, with associated drive electronics andimage processing means (not shown), and a wide angle lens 402 arrangedsuch that all five fingers will be in its field of view. Sensorapparatus 400 can also include an optical illumination system. Theillumination system includes one or more visible light-emitting diodes(LEDs) 403 which are strobed. In one embodiment, LEDs 403 are strobed insynchronization with the image 401. In another embodiment, LEDs 403 arestrobed when a tap is detected.

In this example implementation, as shown in FIG. 4, sensor apparatus 400is implemented by attaching at least the sensor apparatus 400 to thebottom of a wrist of a user, arranged in a manner such that all of thefingertips of the user are within its field of view. As noted above inconnection with FIG. 2, sensor apparatus 400 also can be implemented atother locations, such as by way of the sensor apparatus 301 (or specialpurposed camera) located at the bottom of a palm of a user.

Alternatively, the illumination system is optional and the specialpurposed camera may utilize ambient light to acquire the image. In thisembodiment, visible light emitting diodes 403 are thus not incorporatedin sensor apparatus 400. Optionally, the illumination system can stillbe included (e.g., by including visible light emitting diodes 403) butthe visible light diodes 403 can be left turned off when the ambientlight is sufficient to acquire the image of the finger combination. Inthis optional embodiment, image sensor 401 can be used to sense theambient light level and processor 202 (FIG. 2) can be used to determinewhether there is sufficient light and to control visible light-emittingdiodes 403 by strobing them when necessary.

In yet another embodiment, the sensor apparatus can be a stereoscopiccamera. This embodiment is similar to the embodiments discussed above inconnection with FIGS. 2-4, except that the sensor apparatus consists oftwo image sensors and two lenses, disposed relatively close to oneanother. When a tap is sensed, both image sensors simultaneously acquireimages of the fingers at slightly different perspectives. In thisembodiment the images captured by the image sensors are processed todetermine landmarks in the two images to produce a singlepseudo-three-dimensional image. This compound image is then furtheranalyzed to determine which finger combination has been tapped as in theabove embodiments.

Alternatively, the special purpose camera discussed above in connectionwith FIGS. 2-4 can be a time-of-flight type camera. In this embodiment,a single image sensor is used. In addition to capturing atwo-dimensional image, the sensor apparatus (or special purpose camera)also records three-dimensional depth information. This compound image isfurther processed to determine which finger combination has been tapped.

In some example embodiments, one or more hardware components (e.g.,components 201, 202, and/or 203 (FIG. 3), and/or components 301, 302,303, 304 or 305 (FIG. 3) or 401, 402 and/or 403 (FIG. 4)) can beincorporated into another computing device. For instance, the componentsmay be integrated into a smart watch, wearable glasses, another type ofwearable device, or the like.

Alternatively, the components may be integrated into a desktop computer,laptop computer, tablet computer, or computer screen in which case thesensor apparatus can be, for example, of a form similar to that of thesensor apparatus 300 or 400 described above, but with at least thesensor apparatus and processor (e.g., processor 202) being housed in adevice that is not worn by the user (e.g., a desktop, laptop, or tabletcomputer).

In yet another example embodiment, the sensing apparatus (e.g.,component 201, 300, and/or 400) and processor (e.g., component 202) arenot worn on the body of the user, but instead are remote from the user.For example, in a classroom or office, the sensor apparatus andprocessor may be installed in a centralized fixture, and tap data may becommunicated by a transmitter (e.g., transmitter 203) in the centralizedfixture to respective computing devices being operated by multiple usersin the room. One example embodiment of such a system employs astereoscopic camera and an IR illumination system, such as a scanning IRlaser, that work together to capture images and determine the movementsof users throughout the room.

Feature extraction software is then employed to identify within thefield of view each user and their respective hand and finger locations.For example, after a user activates their respective computing device,the camera or imaging system of the centralized fixture identifies thefinger positions of the user. Accuracy of the sensed finger positionscan be enhanced by utilizing the IR illumination system. The fixtureacquires the finger positions using the IR laser, and acquires images ofeach finger as it is illuminated by the laser. The processor of thefixture determines as each tap is performed whether each of the involvedfingers is in contact with the surface when the tap is sensed, and thenuses a mapping (e.g., as described in further detail below) to determinewhich character or command was input. A transmitter (e.g., component203) of the fixture then transmits to the computing device of theassociated user a message representing the appropriate character.

FIG. 5 illustrates another example system 500 that may be employed inaccordance with example embodiments herein. This example embodimentemploys mechanical sensors 501-1, 501-2, 501-3, 501-4, and 501-5(collectively, 501), such as accelerometers, vibration sensors, tactilesensors, force sensors, pressure sensors, gyroscopes, and the like, tosense the mechanical quantities associated with corresponding fingerssuch as acceleration, vibration, force, pressure and the like. When, forexample, a sudden change in mechanical quantity such as acceleration,vibration, force, pressure and the like, is sensed by any one or more ofthe sensors 501, a microcontroller 502 reads the output of therespective sensor(s) 501 and processes this information.

It should be understood that microcontroller 502 can include a processorsuch as the processor 202 discussed above with respect to FIG. 2. Inaddition, instead of using a microcontroller having input ports (e.g.,analog-to-digital inputs ports), a processor in conjunction with similarperipheral integrated circuitry can be used (e.g., an analog-to-digitalintegrated circuit) and still be within the scope of the invention.Similarly, processor 202 discussed above in connection with FIG. 2 canbe replaced with a microcontroller and still be within the scope of theinvention.

In this embodiment, the sensors 501 are worn in such a way that eachsensor 501 is in contact with a respective finger of a hand.

In this example embodiment, system 500 monitors the outputs of sensor501. The beginning of a tap event, in this situation, is indicated by achange in output from any of the sensors 501 above a pre-determinedthreshold.

Once a tap event is sensed, microcontroller 502 reads data correspondingto each of the sensors 501 for a predetermined period of time (e.g. 60ms), and analyzes the data to determine which of the fingers associatedwith the sensors 501 has made contact with a surface (resulting in asensed finger combination). The microcontroller 502 then selects thecharacter associated with the sensed finger combination and causes atransmitter 503 to transmit the character to a computing device.

In an example embodiment, microcontroller 502 sends the raw data fromthe sensors to a smartphone or other computing device, and the procedureof analyzing the raw data to determine which finger combination has beentapped may be performed within the smartphone or computing device.

Alternatively, the processor 502 extracts certain features from the rawdata, such as peak amplitude, pulse width, rise time, time of arrival or(in the frequency domain) the power density for each of the sensors 501.Such feature information may be transmitted to a smartphone or otherdevice with computing means. Further analysis of these features wouldthen be performed within the smartphone or other computing means todetermine the finger tap combination.

FIG. 6 illustrates an example sensor apparatus 600, which may furtherrepresent the sensor apparatus 500 that is described above in connectionwith FIG. 5. As shown in FIG. 6, mechanical sensors 501 (e.g., such asaccelerometers, vibration sensors, tactile sensors, force sensors,pressure sensors, gyroscopes, and the like) are integrated inside thesensor apparatus 600, which in this example embodiment is in the form ofa finger separator 601 that is flexible. The sensors 501 are located inthe sensor apparatus 600 such that each sensor 501-1, 501-2, 501-3,501-4, 501-5 is substantially in contact with a respective finger thatis inserted within each finger hole 603-1, 603-2, 603-3, 603-4, 603-5 offinger separator 601 along the proximal phalanges (e.g., one sensor perproximal phalanx). Optionally, the sensor apparatus 600 can beconstructed so as to house processor 502 and/or transmitter 503 as well,as shown in FIG. 6. Other components (e.g., power circuitry) can beintegrated within the sensor apparatus 600 as well. The sensors 501 canbe connected to the processor 502 via a bus placed within the structureof the finger separator 601. Alternatively, just the sensors 501 can behoused within the sensor apparatus 600 and connected via a connector(not shown) to the processor 502 and transmitter 503.

In yet another embodiment, just the sensors 501 and processor 502 can behoused (or encased) within the sensor apparatus 600 and connected to aremote transmitter 503 through a connector (not shown). It should beunderstood that the sensors 501 can be incorporated into a form factorother than the finger separator form factor shown in FIG. 6 and still bewithin the scope of the invention. Indeed, the form factor can betailored to different ergonomics for different sizes and shapes offingers and/or hands as will now be described.

This could be accomplished by incorporating the sensors 501 into a glove(or fingerless glove), so that each sensor 501 is in contact with one ofthe fingers along the phalanges. Alternatively, the sensors 501 could beworn across the back of the palm, so that the sensors 501 are in contactwith the fingers along the metacarpal bones of each finger.

System 500 can be integrated into a variety of wearable structures. Inanother example embodiment, one or more system 500 components, forexample, mechanical sensors 501, are incorporated into a flexible pieceof fabric, such as a wearable wristband which can further be wrappedaround the fingers or palm (fiducial marks), and operate in mannerssimilar to those described in the context of the various exampleembodiments described elsewhere herein. The fabric may be comprised ofelastic or another flexible material, in which the circuitry of thesystem 500 components is incorporated using flexible circuit technology.The fabric may be worn by the user by being wrapped around the user'shand, wrist, or other body part, which the user may move to causeparticular characters to be transmitted or inputted to a computingdevice. In this embodiment, several mechanical sensors can be arrangedwithin the layers of the material such that one or more mechanicalsensors measures a mechanical quantity associated with a correspondingfinger when the finger is tapped against a surface.

In another embodiment, the sensor apparatus 501 may be one or moreacoustic sensors each of which senses the sound waves produced when eachfinger contacts the surface. In such a configuration, a single sensor ora multiplicity of sensors can be arranged to sense the acoustic signalsproduced by each tap. The processing means analyzes these signals todetermine which of the fingers contributed to the acoustic signal inorder to determine which finger combination was tapped.

In another embodiment, one or more EMG (electromyography) sensors may beemployed. In this embodiment, sensors are in contact with the hand,wrist or arm of the user, and sense the electrical signals which areproduced by the movement of the fingers and hand. Such a device may alsoemploy a vibration sensor or accelerometer to sense a tap event. When atap occurs the processor analyzes the signals from the EMG sensors tocalculate the relative position of the fingers in order to determinewhich tap combination has been produced.

In another embodiment, Radio Frequency waves may be employed todetermine the relative location of the fingers. In this embodiment, oneor more RF sources produce a brief signal, and a multiplicity of sensorsmeasures the pattern of reflection of such signal. By analyzing thereflection pattern, the location of the fingers is calculated and thetap combination is determined.

In another embodiment, processor 202 (FIG. 2) can be used instead ofmicrocontroller 502 or conversely microcontroller 502 can be usedinstead of processor 202. Additionally, both systems 200 and 500 andtheir respective implementations can be combined. In this embodiment,placement of the camera-based sensor apparatus 201 and mechanicalsensors 501 can be arranged as described herein.

Having described example systems that may be employed in accordance withvarious example aspects herein, reference will now be made to FIG. 7 todescribe an example procedure 700 that may employ such systems.

At block 701, an optional calibration procedure is employed in order tospeed and simplify processing, and to make the detection process morerobust. To do this, a reference signal is first generated by tapping allof the fingers of a hand on a surface, enabling capture and processingof a corresponding reference signal. Since it is known that thereference tap is performed with all fingers in contact with the surface,the calibrated signal would serve as a reference signal that representsthe signal produced by the sensor in a situation in which each of thefingers is in contact with the surface. The device can employ thereference signal as the basis for interpreting future taps, for instanceby determining whether any given fingertip is present in any of thereference “down” positions.

The image sensor apparatus system 200 discussed above in connection withFIGS. 2-4 can be employed to acquire a reference signal. In instanceswhere the sensor apparatus is as discussed above, the reference signalis an image which presents the position of each of the fingers when theyare in the ‘tap’ position. This reference image can be compared tosubsequently acquired images to determine which fingers are shown to bein the ‘tap’ position or the ‘no tap’ position.

The mechanical sensor apparatus system 500 discussed above in connectionwith FIGS. 5 and 6 can also be employed in a manner to acquire areference signal. In such an embodiment, the signals acquired from thesensors from the reference ‘tap’ are analyzed and features are extractedfrom the sensors 501 of each finger which represent the signals producedwhen each of the fingers contacts the surface. This information can becompared with subsequent signals from subsequent tap events todistinguish which fingers have contacted the surface and which have not.

At block 702, a tap is sensed and a determination is made as to whichfinger combination was employed during the tap. In the cameraimplementation, when the tap occurs, an image (or sequence of images) ofthe fingers is acquired and stored in a memory. In the mechanical sensorembodiment, the forces of each mechanical sensor measurement is acquiredand stored in a memory. A processor (e.g., processor 202 or 502) thenexecutes a series of filter algorithms on the acquired images toascertain the locations of the fingertips during the tap. For example,the processor may first execute an edge detection algorithm, such as aCanny Filter, on the acquired image. The result of such a procedure mayinclude a table of pixel locations of the lowest part of each fingertipwithin the image.

At block 703 a determination is made (e.g., by processor 202 or 502)based on the sensed finger combination and the mapping provided by thepresently active keyboard, whether the sensed finger combinationcorresponds to a toggle command. If the processor determines that thesensed finger combination does not correspond to the toggle command,then control is passed to block 705. If, on the other hand, adetermination is made in block 703 that the sensed finger combinationdoes correspond to the toggle command, then at block 704 the processortoggles the keyboard, activating either the default keyboard or thealternate keyboard based on which one of the keyboards is presentlyactive. Control is then passed to block 705.

Once a determination has been made as to which fingers were in contactwith the surface during the tap, the character corresponding to thatfinger combination is calculated based on a look-up table. Inparticular, at block 705, the mapping provided by the active keyboard(e.g., the default keyboard or the alternate keyboard, as shown forexample in Table 1 (below), which in some example embodiments ispre-stored in memory) is employed to map the sensed finger combinationto a corresponding character (a mapped or decoded character).

At block 706, the processor causes the transmitter to transmit themapped or decoded character to the computing device (e.g., a mobile ordesktop computing device) by way of a wired or wireless interface. Atthat point, the system can optionally return to the low-power mode untila next tap event is detected.

FIG. 8 illustrates an example acquired image 801 and a correspondingedge-detected image 802 that may result from executing theabove-described procedure. The processor applies a feature extractionalgorithm, such as a Hough Transform, to the edge-detected image inorder to identify the finger combination of the tap. The fingertips, inone example, are modeled as simple circles (or semi-circles) for theHough Transform. In this manner, the feature extraction process issimplified because of the a priori knowledge of the morphology of thehand, i.e., that there are five fingers, and that the fingers arearranged vertically throughout the image.

Using the five fingers of one hand, a user can tap 31 unique fingercombinations. These finger combinations are mapped to standard ASCIIcharacters. For example, in the English language, the 31 fingercombinations would suffice to represent the 26 letters of the alphabet,and the five remaining combinations could be used to represent common“control” characters, such as SHIFT, RETURN, and BACKSPACE. As describedbelow, one of the finger combinations would represent a “toggle”command, which would enable an alternate character set or sets. Thesealternate character sets can contain additional characters such asnumbers, punctuation and other ASCII characters.

As described above, the processor need not determine which of thefingertips were in contact with the surface (i.e., perform the algorithmthat maps the finger combinations to standard ASCII characters) andinstead can cause the raw data to be transmitted to a computing devicethat is communicatively coupled with the processor (e.g., processor 202or 502) through, for example, a transmitter (e.g., 203 or 503). Thiswould allow for example for the processing device of the computingdevice (e.g., in a smartphone or other computing device) to determinewhich of the fingertips were in contact with a surface during the tap,and which were not.

Table 1 shows an example of how the printable ASCII characters andcommon control characters could be mapped into finger tap combinations.In particular, Table 1 shows which default keyboard character and whichalternate keyboard character corresponds to each finger combination.Finger combinations are identified in the chart by a string of fivecharacters corresponding to the five fingers, respectively, of a user'shand. For instance, the left-most one of the five characters maycorrespond to a user's thumb on their right hand or the user's pinky ontheir left hand, and so on. The right-most one of the five charactersmay correspond to a user's pinky on their right hand or their thumb ontheir left hand, and so on. In each finger combination an X represents afinger that is involved in a tap, and an O represents a finger that isnot involved in the tap, thereby forming a binary number that can bemapped to different characters.

TABLE 1 CHARACTER MAP EXAMPLE DEFAULT ALTERNATE FINGER KEYBOARD KEYBOARDCOMBO. No Shift Shift No Shift Shift 1 XOOOO A A 1 2 OXOOO E E 2 3 XXOOON N 6 4 OOXOO I L 3 5 XOXOO S S ? 6 OXXOO T T 7 7 XXXOO SHIFT SHIFTSHIFT 8 OOOXO O O 4 9 XOOXO K K , 10 OXOXO M M ( ) 11 XXOXO J J - _ 12OOXXO L L 8 13 XOXXO X X / \ 14 OXXXO DELETE DELETE DELETE 15 XXXXO R R. 16 OOOOX U U 5 17 XOOOX Y Y ′ ″ 18 OXOOX G G : 19 XXOOX B B ! 20 OOXOXF F < > 21 XOXOX W W # * 22 OXXOX Q Q + = 23 XXXOX Z Z {circumflex over( )} ~ 24 OOOXX d D 9 25 XOOXX C C $ % 26 OXOXX P P @ & 27 XXOXX V V [ ]28 OOXXX TOGGLE TOGGLE TOGGLE 29 XOXXX RETURN RETURN RETURN 30 OXXXX H H0 31 XXXXX SPACE SPACE SPACE

For convenience, the following example is provided in a case where auser's right hand is being employed for taps. At the start, a defaultkeyboard (or character set) is active. As one example, to tap the letter“a,” the user may momentarily tap their thumb on a surface, in whichcase a sensor apparatus determines that the thumb had been in contactwith the surface at the moment of the tap, and the letter “a” istransmitted to the computing device. To tap a capital “A,” the userwould first tap with their thumb, first, and middle fingerssimultaneously on a surface. This would be interpreted by the sensorapparatus and/or processor as the “SHIFT” command. Then, the user wouldtap the “A” as before, with their thumb.

As mentioned above, to activate an alternate keyboard, for instance tofacilitate selection of a special character, the user would first tapthe “TOGGLE” command, using the middle, ring and pinky fingers. Thiswould activate the alternate keyboard. The user could then tap thefinger combination that corresponds to the number or character desired.Note that by using the TOGGLE command in conjunction with othercombinations, any number of alternative keyboards can be selected.

In one example embodiment herein, to increase the ease and speed bywhich a user can be trained to employ a particular mapping of fingercombinations to characters, finger combinations can be categorized intofinger sets, or categories of finger combinations that share similarcharacteristics. For instance, Table 2 shows example categories orfinger sets.

TABLE 2 Finger Set (Number of possible combinations) Possible FingerCombinations All Fingers Down (1) XXXXX One Finger Down (5) X0000 0X00000X00 000X0 0000X One Finger Up (5) 0XXXX X0XXX XX0XX XXX0X XXXX0 TwoFingers, Together (4) XX000 0XX00 00XX0 000XX Two Fingers, Skip One (3)X0X00 0X0X0 00X0X Two Fingers, Far Apart (3) X000X X00X0 0X00X ThreeFingers Together (3) XXX00 0XXX0 00XXX Three Fingers, Skip Two X00XX andTwo Skips (3) XX00X X0X0X Three Fingers, Skip One (4) X0XX0 (Thumb Down,First Up) XX0X0 (Thumb Down, Middle Down) 0X0XX (Pinky Down, MiddleDown) 0XX0X (Pinky Down, Ring Up)

By grouping finger combinations into sets, and then mapping a finger setto characters that share a common characteristic, a user may find iteasier to learn to use the mapping for character entry. For example,there are five finger combinations possible in the set referred to as“one finger down.” Those five finger combinations may be mapped to thefive vowels of the English alphabet (i.e., a, e, i, o, and u), therebymaking it easier for a user to employ such a mapping.

In another example, which is available to all the embodiments describedherein, the sensor apparatus and/or other components are incorporatedinto a wearable device, and are configured to sense user motion inaddition to finger taps. In response to such sensing, the wearabledevice may facilitate screen-navigation functions on a screen that iseither incorporated into the wearable device or separate from thewearable device. In this manner, the device may utilize the sensed handmotions to control the location of a cursor on a computer screen, thusemulating the function of a mouse. In this situation, certain fingercombinations can be mapped to common mouse commands (e.g. right click,left click).

The connectivity to a separate device can be according to HumanInterface Device Profile (HID), which defines the protocols, proceduresand features to be used by Bluetooth HID enabled devices such askeyboards, pointing devices, gaming devices and remote monitoringdevices.

Alternatively, and/or additionally, the sensor apparatus within thewearable device can be configured to interpret gestures as well as taps.In this case, the device may be used to manipulate objects on a screenor send commands to a computing device.

As can be appreciated in view of the above, the example embodimentsdescribed herein provide a one-handed, chorded data entry system thatenables an operator to tap a combination of fingers on a surface,thereby causing a character to be entered in an electronic device.Unlike previous systems, the system disclosed herein allows the user totap on any surface, with the fingers arranged in any position. Alsounlike previous systems, with this system neither the fingers nor thefingertips of the user need be aimed at any target. As a result, thesystem described herein reduces finger travel and facilitates fast andaccurate data entry.

It should also be understood that the number of fingers required to mapto a finger combination can be fewer than five. This embodiment would beuseful, for example, in the case where a user's hand has fewer than fivefingers due to, for example, an injury or disability.

The following embodiments relate to one-handed, chorded data entrysystems such as those described above in connection with FIGS. 2, 5, 6,and 7, where the data entry system is worn on the phalanges of a hand.As described above, the data entry system incorporates sensors whichdetect finger taps. A sensor apparatus detects the sensor output andcommunications the detected output to a processor which decodes the tapsinto alphanumeric characters. A transmitter transmits the characters toa receiving device as described in any one of the embodiments describedabove.

As shown in FIG. 9, the system can also include plural actuators. Whenthe device is worn by a user, each actuator is in mechanicalcommunication with a corresponding finger. As shown in FIG. 10, thesystem according to this embodiment also includes a receiver 908(optionally in the form of an independent receiver or as part oftransceiver as shown) that receives alphanumeric data from, for example,a computing device 1002 or other device like 904 (e.g., 1104 worn byanother person as shown in FIG. 11). Referring back to FIG. 10, thesystem further includes a decoder 907 that decodes the alphanumericcharacters received via receiver 908 into tap codes. In an optionalembodiment, the operations performed by the decoder 907 are incorporatedin processor 902. The processor 902, in turn, actuates the actuators ofactuator apparatus 906 according to the decoded alphanumeric character.The actuation of the actuators causes the actuators to apply amechanical stimulus to respective fingers. In one embodiment a userselects whether to the mechanical stimulus is applied to the respectivefingers simultaneously or sequentially. This mechanical stimulus modecan be controlled through a mode selector switch on the mechanicalstructure of the system, through a predetermined tap command, through aremote device application, and the like.

If necessary, the processor 902 can be configured to actuateintermediary circuitry (e.g., current or voltage amplifiers, or othersignal conditioning components, such as D/A 909), that is necessary todrive the actuators of actuator apparatus 906. Each time alphanumericdata representing one or more characters is received via the receiver908, the processor 902 decodes the alphanumeric data and activates theactuators which are associated with that character, which in turn, causethe actuators to press against or otherwise make contact with thefingers correspondingly so that the character(s) is/are sensed throughthe fingers (e.g., phalanges).

Using this device, a person can communicate with, for example, anotherdevice 1104 operated by a person (e.g., person 2) as shown in FIG. 11,or computing device 1002 executing, for example, interactiveapplications stored in an application store 1002 a (FIG. 10) such asgame applications, news applications, email applications, and the like)at a distance by tapping their fingers, and receive information throughthe mechanical communication received from the actuators. Thiscommunication can be in real-time, in which a tap from one user istransmitted and sensed by another tap device having a receiver asdescribed above with relatively little (i.e., unperceivable) delay.Alternatively (or optionally) the tap device can be placed in an offlinemode, in which alphanumeric characters that are tapped into the device904 are composed (i.e., sensed), stored in a memory (e.g.,communicatively coupled to the processor; not shown) and sent to bedecoded by a receiving device having actuators as described above at alater time.

In one embodiment, the tap device is housed in a flexible substrate suchas a soft, compressible foam or a fabric. The sensors consist of 3-axisaccelerometers which are coupled to the fingers via rigid weights. Theactuators consist of linear actuators such as micro-solenoids orpiezo-electric linear actuators each of which generate a mechanicalpulse when activated. The mechanical pulse is sensed by a correspondingfinger. In an optional embodiment, the actuators consist of low levelneuromuscular stimulation devices that produce electric pulses that canbe sensed by the fingers, instead of mechanical actuators.

Software embodiments of the example embodiments presented herein may beprovided as a computer program product, or software, that may include anarticle of manufacture on a machine accessible or machine readablemedium having instructions. The instructions on the non-transitorymachine accessible machine readable or computer-readable medium may beused to program a computer system or other electronic device. Themachine or computer-readable medium may include, but is not limited to,floppy diskettes, optical disks, CD-ROMs, and magneto-optical disks orother type of media/machine-readable medium suitable for storing ortransmitting electronic instructions. The techniques described hereinare not limited to any particular software configuration. They may findapplicability in any computing or processing environment. The terms“computer-readable”, “machine accessible medium” or “machine readablemedium” used herein shall include any medium that is capable of storing,encoding, or transmitting a sequence of instructions for execution bythe machine and that cause the machine to perform any one of the methodsdescribed herein. Furthermore, it is common in the art to speak ofsoftware, in one form or another (e.g., program, procedure, process,application, module, unit, logic, and so on) as taking an action orcausing a result. Such expressions are merely a shorthand way of statingthat the execution of the software by a processing system causes theprocessor to perform an action to produce a result.

Portions of the example embodiments of the invention may be convenientlyimplemented by using a conventional general purpose computer, aspecialized digital computer and/or a microprocessor programmedaccording to the teachings of the present disclosure, as is apparent tothose skilled in the computer art. Appropriate software coding mayreadily be prepared by skilled programmers based on the teachings of thepresent disclosure.

Some embodiments may also be implemented by the preparation ofapplication-specific integrated circuits, field programmable gatearrays, or by interconnecting an appropriate network of conventionalcomponent circuits.

Some embodiments include a computer program product. The computerprogram product may be a storage medium or media having instructionsstored thereon or therein which can be used to control, or cause, acomputer to perform any of the procedures of the example embodiments ofthe invention. The storage medium may include without limitation anoptical disc, a Blu-ray Disc, a DVD, a CD or CD-ROM, a micro-drive, amagneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM,a flash memory, a flash card, a magnetic card, an optical card,nanosystems, a molecular memory integrated circuit, a RAID, remote datastorage/archive/warehousing, and/or any other type of device suitablefor storing instructions and/or data.

Stored on any one of the computer readable medium or media, someimplementations include software for controlling both the hardware ofthe general and/or special computer or microprocessor, and for enablingthe computer or microprocessor to interact with a human user or othermechanism utilizing the results of the example embodiments of theinvention. Such software may include without limitation device drivers,operating systems, and user applications. Ultimately, such computerreadable media further includes software for performing example aspectsof the invention, as described above.

Included in the programming and/or software of the general and/orspecial purpose computer or microprocessor are software modules forimplementing the procedures described above.

While various example embodiments have been described above, it shouldbe understood that they have been presented by way of example, and notlimitation. It is apparent to persons skilled in the relevant art(s)that various changes in form and detail can be made therein. Thus, theinvention should not be limited by any of the above described exampleembodiments. Also, as used herein, the singular forms “a,” “an,” and“the,” are intended to include the plural forms as well, unless thecontext clearly indicates otherwise.

In addition, it should be understood that the figures are presented forexample purposes only. The architecture of the example embodimentspresented herein is sufficiently flexible and configurable, such that itmay be utilized and navigated in ways other than that shown in theaccompanying figures.

Further, the purpose of the Abstract is to enable the U.S. Patent andTrademark Office and the public generally, and especially thescientists, engineers and practitioners in the art who are not familiarwith patent or legal terms or phraseology, to determine quickly from acursory inspection the nature and essence of the technical disclosure ofthe application. The Abstract is not intended to be limiting as to thescope of the example embodiments presented herein in any way. It is alsoto be understood that the procedures described herein need not beperformed in the order presented.

What is claimed is:
 1. A data receiving system, comprising: a pluralityof actuators configured to be in mechanical communication with arespective plurality of fingers, each of the plurality of actuatorsarranged to apply a single pulse to a corresponding respective one ofthe plurality of fingers; a receiver operable to receive datarepresenting one or more alphanumeric items; a decoder operable todecode the received data corresponding to each alphanumeric item into atap code from among a set of tap codes, wherein each tap code in the setof tap codes represents a unique predetermined combination of one ormore fingers that corresponds to a particular alphanumeric item, whereinat least one tap code in the set of tap codes represents a combinationof more than one finger, and wherein each tap code in the set of tapcodes represents a combination of five or fewer fingers; and a processorthat controls the actuators to apply, for each tap code from thedecoder, a single pulse to each of the one or more of the plurality offingers that correspond to the tap code, wherein the processor isconfigured to control the actuators to apply a single pulse to each ofthe one or more of the plurality of fingers that correspond to a tapcode simultaneously.
 2. The data receiving system according to claim 1,wherein each alphanumeric item corresponds to one of an alphanumericcharacter, a word, a phrase, and a sentence.
 3. The data receivingsystem according to claim 1, further comprising: a wearable structureconstructed to support the plurality of actuators on a respectiveplurality of phalanges of a hand.
 4. The data receiving system accordingto claim 1, wherein the actuators apply one of a mechanical pulse and anelectrical pulse.
 5. A data receiving method for a system having aplurality of actuators in mechanical communication with a respectiveplurality of fingers, each of the plurality of actuators being arrangedto apply a single pulse to a corresponding respective finger,comprising: receiving data representing one or more alphanumeric items;decoding the received data corresponding to each alphanumeric item intoa tap code from among a set of tap codes, wherein each tap code in theset of tap codes represents a unique predetermined combination of one ormore fingers that corresponds to a particular alphanumeric item, whereinat least one tap code in the set of tap codes represents a combinationof more than one finger, and wherein each tap code in the set of tapcodes represents a combination of five or fewer fingers; and controllingthe actuators to apply, for each tap code from the decoder, a singlepulse to each of the one or more of the plurality of fingers thatcorrespond to the tap code, wherein the controlling controls theactuators to apply a single pulse to each of the one or more of theplurality of fingers that correspond to a tap code simultaneously. 6.The data receiving method according to claim 5, wherein eachalphanumeric item corresponds to one of an alphanumeric character, aword, a phrase, and a sentence.
 7. The data receiving method accordingto claim 5, further comprising the step of: supporting the plurality ofactuators on a respective plurality of phalanges of a hand using awearable structure.
 8. The data receiving method according to claim 5,wherein the actuators apply one of a mechanical pulse and an electricalpulse.
 9. A data entry system comprising: a processor; a plurality ofmechanical sensors arranged to generate signals representing contact ofone or more fingers of a hand singly or in simultaneous combinationagainst an arbitrary surface; and a plurality of actuators, each inmechanical communication with a respective one of the plurality offingers, each of the plurality of actuators arranged to apply a singlepulse to a corresponding respective one of the plurality of fingers;wherein the processor is configured to detect, based on the signalgenerated by at least one of the mechanical sensors, contact by at leastone finger with an arbitrary surface, determine a tap code, based on thesignals generated by the plurality of mechanical sensors during apredetermined period of time indicating which of the one or more fingerscontacted the arbitrary surface, wherein the tap code represents aunique predetermined combination of one or more fingers that correspondsto one of a character, a command, and a combination of a character and acommand, map the determined tap code to the corresponding one of acharacter, a command, and a combination of a character and a command,and transmit the mapped character, command or combination to an externaldevice, and wherein the processor is further configured to receive datafrom the external device, decode the received data to generate a tapcode, and perform control of the plurality of actuators to apply, forthe generated tap code, a single pulse to each of the one or more of theplurality of fingers that correspond to the generated tap codesimultaneously.
 10. A data communication method for a system having aplurality of mechanical sensors arranged to generate signalsrepresenting contact of one or more fingers of a hand singly or insimultaneous combination against an arbitrary surface, and a pluralityof actuators each in mechanical communication with a respective one ofthe plurality of fingers, each of the plurality of actuators arranged toapply a single pulse to a corresponding respective one of the pluralityof fingers, the method comprising: detecting, based on the signalgenerated by at least one of the mechanical sensors, contact by at leastone finger with an arbitrary surface; determining a tap code, based onthe signals generated by the plurality of mechanical sensors during apredetermined period of time indicating which of the one or more fingerscontacted the arbitrary surface, wherein the tap code represents aunique predetermined combination of one or more fingers that correspondsto one of a character, a command, and a combination of a character and acommand; mapping the determined tap code to the corresponding one of acharacter, a command, and a combination of a character and a command;transmitting the mapped character, command, or combination to anexternal device; receiving data from the external device; decoding thereceived data to generate a tap code; and performing control to apply,for the generated tap code, a single pulse to each of the one or more ofthe plurality of fingers that correspond to the generated tap codesimultaneously.
 11. A data receiving system comprising: a plurality ofactuators configured to be in mechanical communication with a respectiveplurality of fingers, each of the plurality of actuators arranged toapply a single pulse to a corresponding respective one of the pluralityof fingers; a receiver operable to receive data representing one or morealphanumeric items; a decoder operable to decode the received datacorresponding to each alphanumeric item into a tap code from among a setof tap codes, wherein each tap code in the set of tap codes represents aunique predetermined combination of one or more fingers that correspondsto a particular alphanumeric item, wherein at least one tap code in theset of tap codes represents a combination of more than one finger, andwherein each tap code in the set of tap codes represents a combinationof five or fewer fingers; and a processor that controls the actuators toapply, for each tap code from the decoder, a single pulse to each of theone or more of the plurality of fingers that correspond to the tap code,wherein the processor is configured to set a setting that indicateswhether pulses should be applied simultaneously or sequentially, andwherein the processor is further configured to selectively control theactuators to apply a single pulse to each of the one or more of theplurality of fingers that correspond to a tap code simultaneously orsequentially in accordance with the setting.
 12. A data receiving methodfor a system having a plurality of actuators in mechanical communicationwith a respective plurality of fingers, each of the plurality ofactuators being arranged to apply a single pulse to a correspondingrespective finger, comprising: receiving data representing one or morealphanumeric items; decoding the received data corresponding to eachalphanumeric item into a tap code from among a set of tap codes, whereineach tap code in the set of tap codes represents a unique predeterminedcombination of one or more fingers that corresponds to a particularalphanumeric item, wherein at least one tap code in the set of tap codesrepresents a combination of more than one finger, and wherein each tapcode in the set of tap codes represents a combination of five or fewerfingers; and controlling the actuators to apply, for each tap code fromthe decoder, a single pulse to each of the one or more of the pluralityof fingers that correspond to the tap code, wherein a setting indicateswhether pulses should be applied simultaneously or sequentially, andwherein the controlling controls the actuators to apply a single pulseto each of the one or more of the plurality of fingers that correspondto a tap code simultaneously or sequentially in accordance with thesetting.